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Abstract 

The article recounts in detail the process used to include 
all department faculty members in the design of outcomes 
assessment measures for the management major, basing 
these on broad learning outcomes which the department had 
previously identified. While other literature has described the 
outcomes assessment design process in broad terms, the current 
article relates specific steps used to produce test questions. 

The article incorporates theory from the outcomes assessment, 
human resource development, and the organizational behavior 
literatures. 
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A Ten-Step Process for Creating Outcomes 
Assessment Measures for an Undergraduate 
Management Program: A Faculty-Driven 
Process 

The first push to conduct outcomes assessment in higher 
education came from the U.S. Department of Education in 1 988, 
which began requiring accreditation organizations to gather 
outcomes data. Specifically, accreditation agencies should 
evaluate, "whether an institution or program — (1) Maintains clearly 
specified educational objectives that are consistent with its mission 
and appropriate in light of the degrees or certificates awarded; (2) 

Is successful in achieving its stated objectives" (U.S. Department 
of Education, 2007, 602.1 7). Within three years, The Association to 
Advance Collegiate Schools of Business (AACSB) began requiring 
institutions to demonstrate educational outcomes in order to gain 
and maintain accreditation (Palomba & Palomba, 1999). 

In addition, the passage of the No Child Left Behind Act led to 
an increase in assessment of student learning outcomes in higher 
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education (Hickok, 2006). Like grade schools and 
high schools, colleges and universities in every state 
are devoting financial and human resources to this 
task (Smith, Szelest, & Downey, 2004). Research 
shows that higher education faculty members 
spend the majority of their time teaching (Peterson 
& Wiesenberg, 2006), so it is important to know 
whether students are learning what is taught. At 
the higher education level, many publications 
which guide the outcomes assessment process 
include charts, grids, grading rubrics, goals, etc. — 
each full of words — but these publications do not 
describe the specifics of how these words were 
derived (e.g., Bresciani, 2002). 

For example, the set of books, Assessment of 
Student Learning in Business Schools (Martel I & 
Calderon, 2005) contains many articles on the 
outcomes assessment design process. In the 
set, one article gives some procedural details, 
but it primarily describes a top-down, Dean- and 
curriculum-committee-driven process and does 
not state specifically how all faculty members 
were involved (Anderson-Fletcher, 2005). Other 
articles generalize about gathering student and 
employer opinions, but again, these do not contain 
sufficient detail for their processes to be replicated 
(e.g., Bauer, 2002; Kretovics & McCambridge, 1 999). 
Indeed, most outcomes assessment articles discuss 
theory and generalities but lack discussion of 
specific practices. 

In many areas of research, such as the field of 
human resource development (HRD), a recurring 
theme is the need to bridge the gap between 
research and practice (Torraco, 2005). The primary 
goal of the current paper is to bridge this gap by 
documenting the outcomes assessment creation 
process that was designed to overcome two 
major obstacles. First, there is a lack of published 
guidelines on the specific steps to take to create 
outcomes assessment measures. The current paper 
documents the steps that were taken to design the 
outcomes assessment measures, and these steps 
can be used by other researchers and by practioners 
when creating learning evaluation instruments. 
Second, in the academic department included in 
the study, there has historically been a very low 



level of participation in decision-making by some 
faculty members. This low level of participation 
may have been due to their belief that their input 
had been dismissed as unimportant in earlier 
department projects. The current paper documents 
the steps that were taken to ensure that all faculty 
members were given the opportunity both to 
participate and also to see that all of their input 
was included in the final outcomes assessment 
measures. 

In sum, the goal of the current paper is bridge 
the gap from research to practice by documenting a 
theory-based set of practices that can be used both 
to create outcomes assessment measures and also 
to include all department members in the decision- 
making process. 

Research shows that organizations perform 
better when employees are encouraged to 
participate in decision-making (Goldstein, 2005). 

We used a critical reflection process (Van Woerkom, 
2004) so that the content of each step of the process 
was derived from the content of the previous step. 

In each subsequent step, each faculty member saw 
how his or her input was included in the assessment 
project. No input from any faculty member was 
discarded, which helped maintain a high level of 
participation (Van Woerkom, 2004). Studies show 
that the process for developing evaluations is 
equally as important to participants as the type of 
evaluation developed (Sloman, 2004). The process 
of developing evaluations helps professors rethink 
what they do in their courses and why (Sloman, 
2004). 

The process we used was based on the 
outcomes assessment, organizational behavior, and 
human resource development (HRD) literatures. 

The outcomes assessment literature was used 
as the foundation for creating measures in the 
Management major. The organization behavior 
literature was used as the basis for managing 
teamwork among faculty members. The HRD 
literature was used as the basis for writing reliable 
and valid test questions based on a given body of 
knowledge. 

In the following sections, we begin with a 
description of the College. Next, we discuss the 
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outcomes assessment and related literatures. 

These are the foundation for the third section, 
which is a description of the steps used to write the 
instruments. 

Overview of the College 

The College is a public institution, founded 
in the mid 1800s, which enrolls approximately 
9,100 students. Within the College, the School of 
Management has 29 faculty members and more 
than 1 ,1 00 students enrolled in its six majors, within 
three academic departments: Management and 
Marketing; Accounting and Information Systems; 
and Finance and Economics. The Department of 
Management and Marketing offers two degrees: 
Bachelor of Science in Management and Bachelor 
of Science in Marketing. Within the Management 
major, the department offers four concentrations: 
general management; operations; human resources; 
and international. The Management major is a 
junior- and senior-level program — students may not 
enroll in the Introduction to Management course 
(i.e., the gateway course) until they have attained 
junior-level status. In January, 2006, when writing 
began of the outcomes assessment instruments 
for the Management major, the department had 
1 1 faculty members and 625 students enrolled in 
its two majors (approximately 70% of these were 
enrolled in the Management major). 

Impetus for Conducting Outcomes Assessment 

As a public institution, the College operates 
under the auspices of the State Board, which 
requires the public colleges in the state to conduct 
outcomes assessment of student learning. Each 
academic department has autonomy to set 
learning outcomes for its majors and to design its 
outcomes assessment measures. This is typical of 
the assessment process in higher education in the 
United States (Peterson & Augustine, 2000b). 

The College supports the development of the 
outcomes assessment process and measures by 
giving each academic department the equivalent 
of one course of release time (three credits) to be 
distributed to one or more faculty members, as 
the department deems appropriate. The College 
also appointed one of its full professors to the 



position of Assistant Vice President for Academic 
Affairs to be an internal consultant in guiding 
outcomes assessment development, which mirrors 
the approach of other colleges (St. Ours & Corsello, 

1 998; Willard, Dearing, & Belair, 2004). In addition, 
the College hired an external consultant as a 
subject-matter expert in outcomes assessment to 
provide guidance to departments. It is common for 
master's level colleges to have college-wide support 
for assessment (Peterson & Augustine, 2000b). The 
College required each academic department to 
report data on student learning outcomes by July 
1, 2007. Therefore, time was of the essence given 
that the department had only 1 8 months to write 
the measures and to gather, analyze, and report 
data. Time limitations can be beneficial, however, 
by increasing faculty members' involvement in the 
process (Haessig & La Potin, 1999). 

Overview of Development of Broad 
Learning Outcomes 

In October, 2004, the members of the 
Department of Management and Marketing 
participated in a School of Management off-site 
retreat, during which department faculty members 
developed broad categories of learning outcomes 
for the Management major — knowledge areas 
in which department members agreed students 
should be proficient upon graduation (e.g., 
forecasting, ethics). A brainstorming process was 
used (Thompson, 2003), within which all faculty 
members of the department gave suggestions 
regarding potential topics. Six broad areas 
were identified: (a) quantitative analysis and 
decision-making; (b) production and operations 
management; (c) management/interpersonal; 

(d) legal framework of management; (e) financial 
analysis and control; and (f) strategic management. 
Subtopics were suggested for each of these six 
broad areas, and this list of 86 subtopics served as 
the basis of the outcomes assessment measures. 
Although the scope of this list may appear to be 
broad, it was designed to include content from each 
of the four management concentrations mentioned 
earlier (i.e., general, operations, human resources, 
and international). 
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Theory Supporting the Instrument 
Development Process 

The department designated one faculty 
member to be the point person to develop 
the outcomes assessment measures for the 
Management major. (The measures for the 
Marketing major were developed by a different 
point person.) We chose a faculty member — rather 
than an administrator — to lead the process because 
assessment results are more tangible and useful 
to faculty members and the college when faculty 
members lead the process (Haessig & La Potin, 1999; 
Jonson & Calhoun, 2002). The point person's task 
was to translate the 86 subtopics into measures that 
could be administered to students. 

Using Pre-Packaged Tests Versus Creating Program- 
Specific Tests 

Many pre-packaged instruments are available 
for measuring student learning in business school 
programs (AACSB, 2006). We chose, however, 
to write our own measures rather than use pre- 
packaged measures from organizations such 
as Educational Testing Service (ETS) for the four 
reasons provided below. 

Measuring program-specific constructs. Fi rst, 
we wanted to measure the body of knowledge 
we had decided is important for our students' 
post-graduation success based on our 86 learning 
outcomes. Pre-packaged assessments might lack 
generalizability across institutions, meaning they 
might not have equal validity and reliability across 
populations and curriculum (Lohman, 2004). Using 
pre-packaged tests would tell us whether our 
students learned something, but it might not tell 
us whether our students learned what we taught 
(Rotondo, 2005). We created our own instruments 
instead because it is critical for instruments to have 
validity to measure the relevant constructs (Carter, 
2001 ). 

Avoiding teaching to a test. Second, we want to 
be able to use the data gathered from outcomes 
assessment to improve our curriculum to enhance 
our students' learning. We do not want to use 
data to alter our program so that we "teach to a 
test, "which could be the case if we chose a pre- 



packaged test. Research has shown that linking 
curriculum to evaluation helps organizations meet 
their strategic goals by focusing attention on course 
objectives (Allen, 2004; Kirkpatrick & Hawk, 2006). 

Maintaining focus on our college mission. Third, 
we are willing to sacrifice the ability to benchmark 
our outcomes against the outcomes of other 
schools (i.e., external validity) in order to more 
closely measure what is taught in our courses (i.e., 
internal validity) and to maintain the ability to tailor 
our program to meet our College mission. Using 
pre-packaged tests would essentially allow an 
outside organization to determine what content 
areas should be emphasized in our curriculum 
without regard to our College mission. 

Outcomes assessment can be viewed as being 
a form of training evaluation, which seeks to 
determine whether training goals were met (Alvarez, 
Salas, & Garofano, 2004). We assume that content 
areas are taught, and seek only to know whether our 
students learned and retained that content. Class 
sizes are limited to 30 students, and faculty members 
are expected to have one-on-one interactions with 
students and to know which pedagogies are most 
effective to increase students' knowledge, skills, 
and abilities. The results of outcomes assessment, 
in conjunction with our knowledge of our students, 
can guide individual faculty members in making 
changes to their courses if they wish to do so. 
Outcomes assessment data is often used in this 
manner (Peterson & Augustine, 2000a). 

Meeting accreditation standards. Fourth, 
the College is accredited by the New England 
Association of Schools and Colleges (NEASC), 
a regional accrediting agency, which requires 
outcomes assessment, but which does not 
require use of pre-packaged tests (Mundhenk, 

2005; Whittlesey, 2005). Further, the School of 
Management may seek AACSB accreditation, 
which also does not require use of a pre-packaged 
test, but which does require colleges to involve 
all faculty members in the outcomes assessment 
process (Anderson-Fletcher, 2005). It is common 
for master's level colleges to cite accreditation as 
the impetus for assessment (Peterson & Augustine, 
2000b). 
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In the past, accrediting agencies focused on 
measuring educational inputs, such as budgets, 
number of faculty, course content, etc. (Henniger, 
1994). Today, however, there is a greater push 
to measure outputs, that is, student learning, as 
per U.S. Department of Education requirements. 

In conducting outcomes assessment, our goal 
is to measure student performance, not faculty 
performance. It is common for measuring student 
performance to be the case (Peterson & Augustine, 
2000a; St. Ours & Corsello, 1 998). This is but one 
of many possible uses of assessment data from 
which we could choose (Bober & Bartlett, 2004). It 
is, however, an important goal because students 
actively and frequently seek feedback on their 
performance (Kuchinke, 2001). In addition, student 
learning is the primary goal of education, and many 
business programs focus on its measurement within 
the outcomes assessment process (Apostolou, 1 999; 
Cherry & Dave, 1 997; Kretovics, 1 999). 

Potential Difficulties in Gaining Consensus 

As is common in many colleges, faculty 
members were encouraged, but not required, to 
participate in the process (Dooris, 1998). Therefore, 
the department chose to designate a single point 
person in order to expedite the process and to 
diminish the impact of four potential difficulties 
in gaining consensus of department members. It 
is important to be aware of potential sources of 
faculty concerns prior to beginning assessment 
design (Haessig & La Potin, 1999). 

The need for outcomes assessment. First, in 
many academic departments, some faculty 
members wholeheartedly embrace the need to 
conduct outcomes assessment while others are 
opposed (Grunwald & Peterson, 2003; Martell, 

2005). Our department is mandated to conduct 
outcomes assessment by the State Board, and 
using a single point person could ensure that the 
project was completed even if there had been the 
type of opposition within the department which 
could derail the functioning of a committee. In 
addition, the point person could emphasize to 
faculty members the specific benefits to them of 
assessment, which could foster their participation in 
the process (Grunwald & Peterson, 2003). 



Development methods. The second potential 
difficulty is that among those faculty members 
who embrace the need to conduct outcomes 
assessment, some will prefer to use one method 
to create the measures (e.g., brainstorming) 
while other faculty members will prefer to use an 
alternate development method (Camacho & Paulus, 
1995). Using a single point person could ensure 
that all department members help to design the 
outcomes assessment measures while avoiding 
the potential difficulty that large amounts of time 
would be used to design the outcomes assessment 
creation process. As mentioned, we used a critical 
reflection process. This was conducted in a method 
similar to the Delphi technique (Van de Ven & 
Delbecq, 1 974). Members were given written 
questions and asked to respond in writing. These 
responses were compiled and resubmitted to 
members. A meeting of the entire group was held 
only at the end of the process. 

Types of outcomes. The third potential difficulty 
is faculty members at many colleges often disagree 
about the type of outcome to be measured, such 
as facts versus skills (Rotondo, 2005). For example, 
one college struggled with this issue when creating 
measures of interpersonal skills (Bommer, Rubin, & 
Bartels, 2005). Specifically, they wondered whether 
assessment should measure the facts related to 
interpersonal skills or the use of interpersonal skills. 
Another college chose to measure both knowledge 
and skills in its marketing program (Davis, Misra, & 
Van Auken, 2002), 

In employee training, we label these outcomes 
as knowledge (i.e., facts) and behavior (i.e., applying 
the facts to behavior on the job), according to 
Kirkpatrick's (1998) model of training evaluation. 

The U.S, Department of Labor (2006) emphasizes 
that successful job performance requires employees 
to possess three characteristics: knowledge (i.e., facts 
and principles); ability (i.e., ability to learn, improve 
or reason); and skill (i.e., ability to perform a task). 

In the test development literature, two 
well-known taxonomies of learning outcomes 
were developed by Bloom (1 956) and Gagne 
(1 985). Bloom's taxonomy includes knowledge, 
comprehension, application, analysis, synthesis, and 
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evaluation. Gagne's taxonomy includes intellectual 
skills, cognitive strategies, verbal information, motor 
skills, and attitudes (Shrock & Coscarelli, 1 996). 

Given that there is a wealth of outcome types 
from which to choose, using a single point person 
could ensure that all appropriate outcome types 
are measured by incorporating ideas from each 
department member. 

Type of assessment. The fourth potential 
difficulty is that faculty members often disagree 
about the type of assessment method that should 
be used (e.g., multiple-choice, case study, oral 
presentation). Studies show that the type of 
outcome measured should dictate the type of 
assessment used (Melancon & Williams, 2006). 

In addition, assessments should use multiple 
assessment methods for results to be reliable and 
valid (Melancon & Williams, 2006; Riggio, Mayes, 

& Schleicher, 2003). Again, using a single point 
person could ensure that all necessary assessment 
methods are used by incorporating ideas from each 
department member. 

In summary, although on its face using a single 
point person seems to imply that the measures 
were designed unilaterally — and could therefore 
lack comprehensiveness — in the current case, using 
a single point person ensured that each faculty 
member's input was given equal weight and that 
the measures were constructed in a timely manner. 

Specific Steps Used to Create the 
Instruments 

In this section, we describe the process that was 
used to meet the two goals of the project: create 
outcomes assessment measures and include all 
department members in the process. The process is 
described in detail to enable other researchers and 
practioners to mimic this process when writing their 
own evaluation instruments. 

The process of translating the 86 subtopics into 
tests questions was conducted between February 
and May 2006. The process consisted of 1 0 major 
steps, most of which invited direct, written input 
from all department faculty members. As stated 
earlier, the two primary goals of this project were 
to create outcomes assessment measures and to 



include all department members in the decision- 
making process. Each of the following 10 steps 
was conducted to meet either one or both of these 
goals. The steps began with a literature review, 
continued with a comparison of the six broad 
learning outcomes and 86 subtopics to course 
syllabi, and ended with writing the test questions. 
These steps are based on guidelines for creating 
outcomes measures in the training evaluation 
literature (e.g., Kirkpatrick, 1998). 

As stated, the College provided release time 
from teaching to the point person to allow time 
to conduct the project, and appointed an internal 
consultant and an external consultant to guide the 
process. The point person met with the internal 
and the external consultants on five occasions from 
February to May to ensure that the project was 
progressing appropriately. 

7. Literature Review 

The point person reviewed relevant literature 
from AACSB, ETS, the U.S. Department of 
Labor, peer-reviewed journals, and relevant 
books regarding outcomes assessment in the 
management discipline, training evaluation, and 
test development. This was done to ensure our 
process and our product are aligned with these 
organizations' standards. A portion of this literature 
was cited above. This phase of the project was 
conducted between February 1 and April 8. 

2. Review of Learning Outcomes 

The point person merged the list of learning 
outcomes that was produced by department faculty 
members during the October 2004 off-site retreat 
with learning outcomes lists that were developed 
during other Management department meetings 
in prior years. This task entailed deleting duplicate 
items and inserting remaining items from one list 
into relevant categories on the other list. The goal 
was to ensure that the list used at the beginning 
of the outcomes assessment project contained all 
the learning outcomes that had been identified. (It 
was assumed that the final outcomes assessment 
instrument would measure only a subset of the 
list, and include only the most critical learning 
outcomes, as per guidance of the internal and 
external consultants.) 
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This resulted in a single list of 86 learning 
outcomes based on the list of 86 subtopics (see 
Appendix). The point person used Microsoft Excel 
to create a grid of the 86 learning outcomes (rows) 
versus the 34 courses required (columns) for the 
management major. This phase of the project was 
conducted between February 5 and 8. 

3. Review of Course Syllabi 

The point person reviewed Management course 
syllabi for the 1 8 Management courses that are 
required for the Bachelor of Science in Management 
to determine which of the 86 learning outcomes 
are taught in which of the 1 8 courses. At this time, 
the list of courses was shortened from 34 to include 
only the 1 8 taught in the Management department 
because the department cannot control what is 
taught in courses offered by other departments, 
such as Math and Accounting. As stated earlier, the 
department is conducting outcomes assessment 
only to measure whether students learned what 
was taught in Management courses. 

An "X" was placed into the grid if a topic was 
covered in a course. In creating a grid, some 
departments use a Likert-type scale ranging from 1 
to 3 to indicate the extent to which a topic is covered 
in a course. In our department, it was not possible 
to determine from syllabi the extent of topic 
coverage, so the grid was limited to indicate only 
whether a topic was covered at all. This phase of the 
project was conducted between February 5 and 8. 

4. Request Course Content Detail from Faculty 
Members 

The point person requested course content 
details from professors whose syllabi lacked the 
required level of specificity. For example, if a 
syllabus listed the topic, "ethics," subtopics could 
include names of specific theories (e.g., Principle 
of the Double Effect) and philosophers (e.g., 
Machiavelli). This information was entered into the 
grid. Most faculty members readily provided detail 
when shown examples of what was requested. 
Faculty members were invited to participate, but 
were not required or pressured to do so. Some 
faculty members chose to refrain from providing 
detail in this stage. This phase of the project was 
conducted between February 9 and 10. 



5. Request Feedback from Faculty Members on the 
Match of Subtopics to Course Content 

The point person distributed copies of the grid 
to faculty members and requested their feedback 
regarding its accuracy in matching content areas 
to specific courses. Nearly all faculty members 
returned their copies of the grid and included 
additional "Xs" or, in rare cases, asked that an "X" be 
deleted from the grid. The point person revised the 
grid as requested and resubmitted it to faculty for 
feedback. This phase of the project was conducted 
between February 1 0 and April 1 . 

6. Condense List of Subtopics 

The point person condensed the list of 
subtopics in the grid. The list was reduced from 
86 to 14 subtopics based upon the criterion that 
a topic should appear in at least three required 
courses. This is to ensure that a student is exposed 
to the topic multiple times to increase long-term 
knowledge retention. The topic could be taught 
in three of the five courses that all Management 
majors must take. Alternately, the topic could be 
taught in two courses that all Management majors 
must take and at least one course in each of the four 
Management concentrations (i.e., general, human 
resources, operations, and international). The grid 
is shown in Table 1 . This phase of the project was 
conducted between April 5 and 6. 

7. Generate List of Facts for Subtopics 

The point person generated a list of facts for 
each of the 14 subtopics based upon course syllabi, 
the point person's own course notes, and relevant 
textbooks. For example, under the subtopic "goal 
setting and planning"facts included "Gantt charts" 
and "PERT charts.'This phase of the project was 
conducted between April 7 and 10. 

8. Request Feedback from Faculty Members on List of 
Facts 

The point person distributed copies of the 
condensed grid in Table 1 and a separate list of 
the 1 4 subtopics and facts for each subtopic to 
department faculty members. Faculty members 
were asked to add and delete facts from the list, and 
most faculty members returned their copies of the 
list with additions. 
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Table 1 

Grid of Outcomes Assessment Content Areas and Required Management Courses 



Courses and Concentrations within the Bachelor of Science Management Degree 



Core content in Management Area 
1 . Quant, analysis & decision-making 
Statistical presentations 



All Concentrations Gen 

*249 301 341 348 461 329 



Gen-HR HR 



320 322 423 424 425 428 



Interna. Operations . 

342 345 335 347 355 455 



2. Production and operations management 
Forecasting x_ 



3. Management-interpersonal 
Strategy formulation x x 

Managerial decision-making x x 

Organizational goal setting & planning x x 

Leadership x x 

Communication processes within org's x x 

Definition and importance of mgt x x 

Managing in a global environment x x 

Managing teamwork & group dynamics x 
History of management thought x x 



x x 



x x 
x 

x 



4. Legal framework of management 
ethics 


X X 


XXX XXX 




5. Financial analysis and control 
None 


6. Strategic management 
Strategic management model 
Strategic mgt in global business env. 


X 

X 


XX XX 

XX X X 


X 



*A/ofe:The numbers correspond to course numbers for management courses. The course names are as follows: 249 Quantitative Business Analysis II; 301 
Introduction to Management; 341 Business, Government, and Society; 348 Operations Management; 461 Management Seminar; 329 Organizational Theory; 
320 Human Resource Management; 322 Organizational Behavior; 423 Compensation and Benefits; 424 Labor Relations; 425 Recruitment and Selection; 428 
Training and Development; 342 Comparative Management; 345 International Business; 335 Process Analysis; 347 Supply Chain Management; 355 Quality 
Assurance; and 455 Strategic Operations Management. 



Faculty members who did not return a list after 
several days were sent an email with the statement, 
"If you would like your input to be included, please 
send your list to me." This statement let faculty 
members know that their participation was strictly 
voluntary, that the project would continue whether 
or not they participated, and that their ideas would 
definitely be included if they did participate. Several 
professors sent in their lists following this request. 

It was critical to have involvement from as diverse 
a group as possible to maximize face validity of the 
instruments (Manyon, Feeley, Panzarella, Servoss, 
2003). In addition, faculty involvement is critical 
to ensuring meaningfulness of the test results 



(Gerretson & Golson, 2004). This phase of the project 
was conducted between April 1 0 and April 1 5. 

9. Write Test Questions for Each Fact 

The point person created 1 4 tests, one for each 
subtopic. Each test contains at least one question 
for each fact that faculty members identified. The 
tests are relatively short, approximately one page 
each, and contain 5 to 20 questions each. An 
example of test questions is shown in Table 2. We 
chose to create 1 4 short tests rather than one large 
test to mirror the concept of testlets (also known 
as scalets). A testlet is a short test measuring a 
single construct that is used to minimize students' 
interpreting questions in the wrong context (Pike, 
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2005). Testlets tend to have high validity and 
reliability, and can be used effectively with small 
samples of students (Pike, 2006). In addition, they 
help faculty members match test results to their 
course material. 

The questions were taken from the point 
person's own final exams and midterms, when 
possible, or were written based on the facts 
identified and based on informal conversations with 
faculty members that had occurred during earlier 
phases of the project. It is crucial that test questions 
be matched to specific facts or skills (Manyon et 
al., 2003). For example, throughout the project, 
several faculty members requested that we measure 
knowledge, skills, and abilities, so questions were 
written to measure each type of learning outcome. 
Table 2 contains a sample of questions regarding 
groups and group decision-making that measure 
knowledge (facts) and abilities (thought processes 
related to groups). This phase of the project was 
conducted between April 15 and May 9. 

70. Request Feedback from Faculty Members on Test 
Questions 

The point person distributed the revised list 
of 1 4 subtopics and facts to faculty members and 
distributed the 1 4 tests to faculty members. All 
faculty members met as a group to discuss the tests. 
Faculty members who expressed concern with the 
topics of some specific questions or with the high 
degree of specificity of questions were reminded 
that each question was based on a specific fact that 
they had identified or approved in a previous step. 
At this meeting, the point person requested that, 
during the summer, faculty members examine the 
questions for validity and reliability, add and delete 
test questions, and return the tests in September. 
This meeting was held on May 1 0. 

Note that this was the first time the department 
members met as a group to discuss the project. 
Between February and May, the point person met 
only informally with individual faculty members to 
discuss the project. Waiting until the test questions 
were written before holding a group meeting to 
discuss the test questions ensured that each faculty 
member's input was given equal weight in the 
content of the test questions. It also ensured that 



Table 2 

Sample of Outcomes Assessment Questions 
for Group Decision-Making 

1. Give four reasons why people join groups. 

2. Name and define four of the five stages of group 
development. 

3. What is an advantage of group decision-making? 

4. What is a disadvantage of group decision-making? 

5. When is better to use a group with 1 0 members 
rather than a group with 3 members? 

6. If a group has high cohesiveness, and its goals are not 
aligned with organization goals, 

what is the effect on group productivity and why? 



faculty members did not become sidetracked from 
the task at hand. For example, early in the project, 
some faculty members wanted to design the logistics 
of outcomes assessment (e.g., types of questions, 
dates of testing, location of testing). The point 
person was able to make note of these concerns for 
future use and to ask that faculty members remain 
focused by saying, "You are on step 9, but I am still 
on step 3. Let's do this step first, and then we can 
figure out that step based on what we do in this 
step." This statement let faculty members know that 
their concerns would definitely be addressed, but at 
a later date. In addition, keeping faculty members 
focused on one step at a time helped to simplify the 
process for them, which is important in maintaining 
participation (Dodeen, 2004). 

Administering the Tests and Remaining Steps 

The instruments were pilot tested in December 
2006. Approximately 1 0 students took each test 
and indicated the courses they had completed. In 
January, 2007, the department faculty members 
were given a report of the test scores and met to 
discuss the results. Most questions that no students 
answered correctly were then deleted because 
faculty members realized that this material was not 
covered in any courses (e.g., the statistical concept 
"kurtosis"), so that even those students who had 
taken the courses could not have given the correct 
answer. The department members decided to 
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meet at a later date to discuss whether this material 
should be added to courses. 

A small percentage of other questions were 
reworded to make their meanings clearer. In 
addition, questions that were deemed too difficult 
because they were two-part questions (e.g., 

"What is a group and why is it used?") were split 
into separate questions (e.g., "What is a group?" 
and "Why are groups used?") The modified tests 
were administered in May 2007 (these tests are 
available from the author upon request). Several 
faculty members suggested that the data would be 
more valid if the sample size were larger, which is 
supported by the literature (Wood & Conner, 1 999). 
Therefore, this time, approximately 20 students 
completed each test, and results were submitted to 
the faculty members and the College administration 
in June 2007. Faculty members met to discuss the 
results in September, 2007. 

The larger sample sizes of the second round 
of testings allowed further analysis of the validity 
and reliability of the tests. For example, we could 
compare test scores across semesters to determine 
whether each sample of students performs equally. 
We were constrained, however, from conducting 
traditional test-retest reliability analysis due to the 
limitations on the amount of time that can be taken 
from classes to conduct outcomes assessment. 

We also could conduct inter-rater reliability 
studies of the test scoring. Internal consistency 
analysis (Cronbach's alpha) would not be 
appropriate for the tests, however, because each 
test measures multiple constructs and because 
question responses are scored dichotomously. 

We intend to conduct additional analysis on the 
tests to compare the test scores with course grades. 
We will compare the mean test scores with mean 
course grades over the previous two semesters. 

For example, we will compare mean scores on the 
statistical tests with mean grades in the statistics 
courses. The comparisons will be guided by the 
course topic content grid mentioned earlier. We 
expect to find a positive correlation between mean 
test scores and mean course grades. 

We have not designed long-term testing 
procedures. For example, we could administer one 



comprehensive test at the end of students' senior 
year, and if so, administer the test to all students or 
to a random sample of students. Alternately, we 
could administer shorter, topic-specific tests and, 
if so, we could embed these into courses or offer 
them at the end of courses. 

We also are debating whether to administer the 
instruments so that scores count toward course 
grades. A body of literature indicates that students' 
motivation to perform well on tests is low if course 
grades are not at stake (Napoli & Raymond, 2004). 
This low motivation negatively affects test scores 
and reliability. If we build the test scores into course 
grades, then we will need to have a larger bank of 
questions that can be periodically rotated to reduce 
the chance of students sharing questions with other 
students. Currently, the students do not receive 
copies of the tests to keep, and we assume the 
likelihood is low that they would remember or share 
the questions with other students. 

Last, we are debating whether to use time-series 
data gathering rather than cross-sectional data 
gathering. Results would be more meaningful if we 
could track changes in students' knowledge over 
time (Astin & Lee, 2003). 

Conclusions 

The current paper documented a process used 
to bridge the gap between research and practice 
in writing outcomes assessment instruments. We 
began with a review of the relevant literature 
(i.e., training evaluation, outcomes assessment, 
organizational behavior, management). Next, 
we designed a 1 0-step process based on this 
literature. This process can be used by other 
researchers and practioners to design training 
evaluation instruments. Using these steps can 
be especially beneficial when input from a large 
group is necessary, when there is opposition to 
the project from a few members, or when some 
members have not participated in previous projects 
because they believe their input had been treated 
as unimportant. 

Limitations of the Project 

Although the department succeeded in creating 
outcomes assessment measures, there were several 
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drawbacks to using the process described. First, 
the process was very time-consuming for the point 
person. It is doubtful that a faculty member would 
be able to manage this process without being 
given release time from other duties. Second, it is 
politically difficult to manage the process. The point 
person will necessarily be torn between wanting to 
discuss the project and obtain oral input whenever 
possible, and refraining from allowing randomly 
held conversations to take precedence over written 
input. 

Third, members of the department can feel 
left out, even when providing written input. This 
is particularly the case in a department in which 
members see each other almost daily and are 
accustomed to discussing department projects. 
Their inability to discuss the project as a group 
might lead some members to feel alienated because 
they were unable to have give-and-take with other 
department members about the project during 
each of the steps. It is noted, however, that in the 
current project, group discussions were avoided to 
ensure that no member of the department could 
dismiss another member's input. 

References 

Allen, J. (2004). The impact of student 
learning outcomes assessment on technical and 
professional communication programs. Technical 
Communication Quarterly, 73(1 ), 93-1 08. 

Alvarez, K., Salas, E., & Garofano, C. M. (2004). An 
integrated model of training effectiveness. Human 
Resource Development Review, 3(4), 383-41 6. 

Anderson-Fletcher, E. (2005). Going from zero to 
sixty in twelve months: Implementing assessment 
at the Bauer College of Business. In K. Martell, &T. 
Calderon (Eds.), Assessment of student learning in 
business schools: Best practices each step of the way, 
Vol. 1 (pp. 64-83). Tallahassee, FL: Association for 
Institutional Research. 

Apostolou, B. A. (1999). Outcomes assessment. 
Issues in Accounting Education, 74(1), 177-197. 

Association to Advance Collegiate Schools of 
Business International. (2006). Assessment Resource 
Center. Retrieved July 1, 2009, from http://www. 
aacsb.edu/resource_centers/assessment/default.asp 



Astin, A. W., & Lee, J. J. (2003). Flow risky are one- 
shot cross-sectional assessments of undergraduate 
students? Research in Higher Education, 44(6), 657- 
672. 

Bauer, K. W. (2002). Two major assessment 
efforts under way at the University of Delaware. 
Assessment Update, 74(6), 5-7. 

Bloom, B. S. (Ed.). (1956). Taxonomy of educational 
objectives. Handbook I: Cognitive domain. New York: 
David McKay. 

Bober, C. F., & Bartlett, K. R. (2004). The utilization 
of training program evaluation in corporate 
universities. Human Resource Development 
Quarterly, 75(4), 363-383. 

Bommer, W. hi, Rubin, R. S., & Bartels, L. K. (2005). 
Assessing the unassessable: Interpersonal and 
managerial skills. In K. Martell, &T. Calderon (Eds.), 
Assessment of student learning in business schools: 
Best practices each step of the way, Vol. 7 (pp. 1 03- 
1 29). Tallahassee, FL: Association for Institutional 
Research. 

Bresciani, M. J. (2002). Development of a rubric 
to evaluate academic program assessment plans at 
North Carolina State University. Assessment Update, 
74(6), 14-15. 

Camacho, L. M., & Paulus, P. B. (1 995). The role of 
social anxiousness in group brainstorming. Journal 
of Personality and Social Psychology, 68, 1 071 -1 080. 

Carter, S. D. (2001). Assessment. In J. Michee (Ed.), 
Reader's guide to the social sciences (pp. 84-85). 
London: Fitzroy Dearborn. 

Cherry, R. L., & Dave, D. S. (1 997). An application of 
outcomes assessments to measure effectiveness of 
graduate courses in the United States. International 
Journal of Management, 74(4), 646-652. 

Davis, R., Misra, S., & Van Auken, S. (2002). A 
gap analysis approach to marketing curriculum 
assessment: A study of skills and knowledge. 

Journal of Marketing Education, 24(3), 218-224. 

Dodeen, FI. (2004). Simplification of the 
assessment process: A key to faculty motivation. 
Assessment Update, 76(4), 7-8. 

Dooris, M. (1 998). Strategies to involve faculty in 
assessment: Penn State. Assessment Update, 70(4), 5, 
12 . 



Page 12 



AIR Professional File, Number 1 13, Creating Outcomes Assessment Measures 



/HR 

Association for 
Institutional Research 



Gagne, R. M. (1 985). The conditions of learning (4 th 
ed.). New York: Holt, Rinehart and Winston. 

Gerretson, H., & Golson, E. (2004). Introducing and 
evaluating course-embedded assessment in general 
education. Assessment Update, 76(6), 4-6. 

Goldstein, M. (2005, July). Building "speak-up" in 
corporations. T&D, 37-42. 

Grunwald, H., & Peterson, M. W. (2003). Factors 
that promote faculty involvement in and 
satisfaction with institutional and classroom student 
assessment. Research in Higher Education, 44(2), 
173-204. 

Haessig, C. J., & La Potin, A. S. (1 999). Lessons 
learned in the assessment school of hard knocks: 
Guidelines and strategies to encourage faculty 
ownership and involvement in outcomes 
assessment. Assessment Update, 7 7(5), 6-7. 

Henniger, E. A. (1 994). Outcomes assessment: The 
role of business school and program accrediting 
agencies. Journal of Education for Business, 69(5), 
296-300. 

Hickok, E. (2006, October 11). No undergrad 
left behind. New York Times. Retrieved June 30, 

2009, from http://www.nytimes.eom/2006/10/1 1/ 
opinion/1 1 hickok.html 

Jonson, J. L., & Calhoun, T. C. (2002). Don't bring 
faculty to assessment, bring assessment to faculty. 
Assessment Update, 14(6) 8-10. 

Kirkpatrick, D. (1998). Great ideas revisited. In D. 
Kirkpatrick (Ed.), Another look at evaluating training 
programs (pp. 3-8). Alexandria, VA: American 
Society forTraining and Development. 

Kirkpatrick, J., & Hawk, L. (2006, June). Curriculum 
and evaluation: Maximizing results. T&D, 61-62. 

Kretovics, M. A. (1 999). Assessing the MBA: What 
do our students learn? The Journal of Management 
Development, 18(2), 125-136. 

Kretovics, M. A., & McCambridge, J. (1999). A 
seven-step approach to developing an outcomes 
assessment program. Assessment Update, 7 7(2), 
10 - 11 . 

Kuchinke, K. P. (2001). Feedback seeking in 
university human resource development education 
in the US, UK, and Singapore. Human Resource 
Development International, 4(1 ), 1 07-1 26. 



Lohman, M. C. (2004). The development of a 
multirater instrument for assessing employee 
problem-solving skill. Human Resource Development 
Quarterly, 75(3), 303-323. 

Manyon, A.T., Feeley,T. H., Panzarella, K. J., & 
Servoss, T. J. (2003). Development of an assessment 
tool measuring medical students' integration of 
scientific knowledge and clinical communication 
skills. Assessment Update, 75(1), 1-2, 14-15. 

Martell, K. (2005). Overcoming faculty resistance 
to assessment. In K. Martell, &T. Calderon (Eds.), 
Assessment of student learning in business schools: 
Best practices each step of the way, Vol. 1 (pp. 210- 
226). Tallahassee, FL: Association for Institutional 
Research. 

Martell, K., & Calderon, T. (Eds.). (2005). Assessment 
of student learning in business schools: Best practices 
each step of the way. Tallahassee, FL: Association for 
Institutional Research. 

Melancon, S. C., & Williams, M. S. (2006). 
Competency-based assessment center design: 

A case study. Advances in Developing Human 
Resources, 8(2), 283-31 4. 

Mundhenk, R.T. (2005). Assessment in the context 
of accreditation. In K. Martell, &T. Calderon (Eds.), 
Assessment of student learning in business schools: 

Best practices each step of the way, Vol. I (pp. 27-42). 
Tallahassee, FL: Association for Institutional Research. 

Napoli, A. R., & Raymond, L. A. (2004). How reliable 
are our assessment data? A comparison of the 
reliability of data produced in graded and un- 
graded conditions. Research in Higher Education, 
45(8), 921-929. 

Palomba, N. A., & Palomba, C. A. (1 999). AACSB 
accreditation and assessment in Ball State 
University's College of Business. Assessment Update, 

7 7(3), 4-5, 15. 

Peterson, M. W., & Augustine, C. H. (2000a). 
Organizational practices enhancing the influence 
of student assessment information in academic 
decisions. Research in Higher Education, 47(1), 

21-52. 

Peterson, M. W., & Augustine, C. H. (2000b). External 
and internal influences on institutional approaches to 
student assessment: accountability or improvement? 
Research in Higher Education, 4 1 (4), 443-479. 



Page 13 



AIR Professional File, Number 1 13, Creating Outcomes Assessment Measures 



/HR 

Association for 
Institutional Research 



Peterson, S. L., & Wiesenberg, F. P. (2006). The 
nature of faculty work: A Canadian and US 
comparison. Human Resource Development 
International, 9(1 ), 25-47. 

Pike, G. R. (2005). Assessment measures: Using 
scalets in surveys of student learning. Assessment 
Update, 77(1), 14-16. 

Pike, G. R. (2006). The dependability of NSSE 
scalets for college- and department-level 
assessment. Research in Higher Education, 47(2), 
177-195. 

Riggio, R. E., Mayes, B. T., & Schleicher, D. J. (2003). 
Using assessment center methods for measuring 
undergraduate business student outcomes. Journal 
of Management Inquiry, 72(1), 68-78. 

Rotondo, D. M. (2005). Assessing business 
knowledge. In K. Martell, &T. Calderon (Eds.), 
Assessment of student learning in business schools: 
Best practices each step of the way, Vol. 7 (pp. 82- 
102). Tallahassee, FL: Association for Institutional 
Research. 

Shrock, S. A., & Coscarelli, W. C. C. (1 996). Criterion- 
referenced test development. Washington, DC: The 
International Society for Performance Improvement. 

Sloman, M. (2004, September). Learning 
evaluation, or not. T&D, 45. 

Smith, J. S., Szelest, B. P., & Downey, J. P. (2004). 
Implementing outcomes assessment in an 
academic affairs support unit. Research in Higher 
Education, 45(4), 405-427. 

St. Ours, P. A., & Corsello, M. (1 998). Faculty-driven 
assessment: A collaborative model that works. 
Assessment Update, 70(4), 6. 



Thompson, L. (2003). Improving the creativity 
of organizational work groups. Academy of 
Management Executive, 77(1) 96-109. 

Torraco, R. J. (2005). Ratings, rankings, results, and 
what really matters. Human Resource Development 
Review, 4(1 ), 3-7. 

U.S. Department of Education. (2007). 
Accreditation in the United States. Retrieved June 
30, 2009, from: http://www.ed.gov/admins/finaid/ 
accred/index.html 

U.S. Department of Labor. (2006). Description 
of knowledge, ability, and skills. Washington, DC: 
Author. Retrieved July 2, 2009, from: http://online. 
onetcenter.org/skills/ 

Van de Ven, A., & Delbecq, A. (1 974). The 
effectiveness of nominal, Delphi, and interacting 
group decision processes. Academy of Management 
Journal, 77,605-621. 

Van Woerkom, M. (2004). The concept of critical 
reflection and its implications for human resource 
development. Advances in Developing Human 
Resources, 6(2), 1 78-1 92. 

Whittlesey, V. (2005). Student learning outcomes 
assessment and the disciplinary accrediting 
organizations. Assessment Update, 77(4), 10-12. 

Willard, W. A., Dearing, F., & Belair, S. J. (2004). 
Facilitating assessment through faculty ownership: 
The assessment liaison system. Assessment Update, 
76(3), 11-13. 

Wood, P, & Conner, J. (1 999). Deciding how 
many participants to use in assessment research. 
Assessment Update, 7 7 (4), 8-1 1 . 



Page 14 



AIR Professional File, Number 1 13, Creating Outcomes Assessment Measures 



/HR 

Association for 
Institutional Research 



Appendix 



Core content in Management Area 
1. Quantitative Analysis & Decision-Making 


Number of Courses* 
All Gen 


HR 


Int 


Ops 


Any 


1 Statistical Presentations 


2 


1 


3 


1 


1 


7 


2 Analysis of Business Data 


2 


0 


0 


1 


1 


4 


3 Describing Business Data 


1 


1 


3 


1 


1 


6 


4 Linear Regression and Correlation Analysis 


1 


1 


2 


0 


0 


3 


5 Multiple Regression and Correlation 


1 


1 


2 


0 


0 


3 


6 Probability and Probability Distributions 


1 


0 


0 


0 


1 


2 


7 Sampling Distributions and Confidence Intervals 


1 


0 


0 


0 


1 


2 


8 Hypothesis Testing with Various Distributions 


1 


0 


0 


0 


0 


1 


9 Analysis of Variance 


1 


0 


0 


0 


0 


1 


II. Production And Operations Mgt 


All 


Gen 


HR 


Int 


Ops 


Any 


10 Forecasting 


3 


0 


0 


1 


1 


5 


11 Inventory Management 


2 


0 


0 


1 


0 


3 


1 2 Capacity and Location Planning 


1 


1 


0 


1 


3 


6 


13 Quality Control 


1 


0 


0 


0 


3 


4 


14 Product and Service Design 


1 


1 


0 


0 


1 


3 


15 Production and Work Systems 


1 


0 


0 


0 


2 


3 


16 Planning and Scheduling Techniques 


1 


0 


0 


0 


2 


3 


1 7 CAD (Computer Aided Design) 


1 


1 


0 


0 


0 


2 


1 8 CIM (Computer Integrated Manufacturing) 


1 


1 


0 


0 


0 


2 


1 9 Historical Devel. of Modern Op's Management 


1 


0 


0 


0 


0 


1 


20 Learning Curves 


1 


0 


0 


0 


0 


1 


21 Productivity and International Competition 


1 


0 


0 


0 


0 


1 


22 Product and Service Design: Tech & Automation 


0 


1 


0 


0 


1 


2 


23 Project Management 


0 


0 


0 


0 


1 


1 


24 Product and Service Design: Reliability, etc. 


0 


0 


0 


0 


0 


0 


III. Management/Interpersonal 


All 


Gen 


HR 


Int 


Ops 


Any 


25 Strategy Formulation 


5 


2 


2 


1 


0 


9 


26 Managerial Decision-Making 


4 


1 


0 


1 


0 


6 


27 Organizational Goal Setting and Planning 


3 


1 


3 


1 


1 


8 


28 Leadership 


3 


1 


2 


1 


1 


7 


29 Communication Processes Within Organizations 


3 


3 


2 


1 


0 


7 


30 Definition and Importance of Management 


3 


1 


1 


2 


0 


6 


31 Managing in a Global Environment 


2 


3 


4 


2 


1 


10 


32 Organization Culture 


2 


3 


5 


1 


0 


9 


33 Career Development Issues 


2 


2 


5 


2 


0 


9 


34 Managing Teamwork and Group Dynamics 


2 


2 


4 


1 


1 


8 


35 History of Management Thought 


2 


2 


3 


1 


1 


7 


36 Persuasion and Impression Management 


2 


2 


4 


1 


0 


7 


37 Managing Innovation and Change 


2 


1 


0 


2 


1 


6 


38 Managing Diversity 


2 


2 


2 


2 


0 


6 


39 The Four Management Functions 


2 


0 


1 


1 


1 


5 


40 Organizational Politics (Power and Conflict Mgt) 


2 


2 


2 


0 


0 


5 


41 Social Responsibility 


2 


1 


0 


2 


0 


5 


42 Self-management 


2 


2 


2 


1 


0 


5 


43 Current Trends (TQM, Self-directed Teams, etc.) 


1 


3 


4 


2 


1 


9 


44 Basics of HR Mgt (Acquisition, Developing, etc.) 


1 


1 


4 


2 


0 


7 


45 Approaches to Motivation 


1 


2 


3 


1 


0 


5 
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46 Managing Individual Differences 


1 


2 


3 


1 


0 


5 


47 Coaching and Developing Others 


1 


2 


3 


0 


0 


4 


48 Fundamentals of Organizing 


1 


1 


0 


0 


1 


3 


49 Management Control Systems 


1 


0 


0 


0 


0 


1 


IV. Legal Framework of Management 


All 


Gen 


HR 


Int 


Ops 


Any 


50 Ethics 


3 


3 


5 


0 


0 


9 


51 Managerial Process and the Law 


2 


0 


0 


2 


0 


4 


52 Consumer Protection 


2 


0 


0 


1 


1 


4 


53 Employment Discrimination Law 


1 


2 


4 


0 


0 


5 


54 Business and Dispute Resolution Processes 


1 


0 


1 


2 


0 


4 


55 Business, the Constitution, and Gov't Agencies 


1 


0 


1 


1 


0 


3 


56 Product Liability and Intellectual Property 


1 


0 


0 


1 


0 


2 


57 Business and the Judicial Process 


1 


0 


0 


0 


0 


1 


58 Labor Relations Law 


0 


1 


3 


1 


0 


4 


59 Law of Contracts, Uniform Commercial Code, etc. 


0 


0 


0 


0 


2 


2 


60 Managing with Tort Laws 


0 


0 


0 


0 


0 


0 


V. Financial Analysis & Control 


All 


Gen 


HR 


Int 


Ops 


Any 


61 Analysis of Financial Statements 


1 


0 


1 


1 


0 


3 


62 Cost of Capital 


1 


0 


0 


1 


0 


2 


63 Cash Flow Management 


1 


0 


0 


1 


0 


2 


64 Risk and Rates of Return 


1 


0 


0 


0 


0 


1 


65 Capital Budgeting 


1 


0 


0 


0 


0 


1 


66 Working Capital Management 


1 


0 


0 


0 


0 


1 


67 Time Value of Money 


0 


0 


0 


0 


0 


0 


68 Bond and Stock Evaluation 


0 


0 


0 


0 


0 


0 


69 Capital Structure and Leverage 


0 


0 


0 


0 


0 


0 


VI. Strategic Management 


All 


Gen 


HR 


Int 


Ops 


Any 


70 Strategic Management Model 


3 


1 


2 


0 


0 


6 


71 Analysis of theTask and General Environment 


2 


1 


3 


0 


0 


6 


72 Strategic Mgt in Global Business Environment 


2 


1 


1 


1 


1 


6 


73 Environmental Scanning and Industry Analysis 


2 


1 


2 


0 


0 


5 


74 Strategy Implementation: Organizing, Staffing, etc. 


2 


1 


2 


0 


0 


5 


75 Hierarchy of Strategy 


2 


1 


1 


0 


0 


4 


76 Internal Organizational Scanning and Analysis 


2 


1 


1 


0 


0 


4 


77 Building Competitive Advantage 


2 


0 


1 


0 


0 


3 


78 Designing Strategy Supportive Control & Rewards 


1 


0 


2 


0 


1 


4 


79 Functional Level Strategies 


1 


1 


1 


0 


0 


3 


80 Business Level Strategies 


1 


1 


1 


0 


0 


3 


81 Value Analysis 


1 


0 


0 


0 


1 


2 


82 Growth Strategies 


1 


1 


0 


0 


0 


2 


83 Designing a Strategy Supportive Organization 


1 


1 


0 


0 


0 


2 


84 Strategy Formulation: Situation Analysis & Strategy 


1 


1 


0 


0 


0 


2 


85 Strategic Management of Small Business 


1 


0 


0 


0 


0 


1 


86 Strategic Decision-makers 


1 


0 


0 


0 


0 


1 



*Note: All = courses that all management majors must take; Gen = courses that students with the general management concen- 
tration must take; HR = human resources concentration; Int = international concentration; Ops = operations concentration; Any = 
total number of course in which a topic appears. 
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